You have a computer. You're afraid of it being stolen by baddies or raided by the police, so you've encrypted its hard disk with LUKS. You also want to carry around some of your crypto keys (SSH and/or GPG), or any kind of sensitive data, on an USB key, so you can restore a normal activity in case the computer gets stolen or becomes untrustworthy. You like things that Just Work, including automounting with autofs (no GUI popups, no need to manually unmount and so on). But of course, you can't very well just mount the drive and store your keys on it in the clear. Well, actually you can (I did it for way too long), but let's assume you want to do without. Because USB keys get stolen or lost, too. autofs just handles the mounting of the drives, not their unlocking, and there's no hook in there specifically for LUKS. There is, however, a more generic way of running arbitrary commands on mounting: the program maps. It's all described in the documentation, so I'll just paste the relevant parts of my /etc/auto.removable, with a few comments. Remember to make it executable and reference it in /etc/auto.master.

 #! /bin/sh
 volume=$1
 autoluks ()  
   dev=/dev/$1
   cryptname=$ 1 _crypt
   cryptdev=/dev/mapper/$cryptname
   keyfile=/etc/private/$cryptname.key
   options=$2
   [ ! -e $dev ] && exit 0
   if [ -b $cryptdev ] && ! cryptsetup status $cryptname   grep -q device:.*$dev ; then
       cryptsetup remove $cryptname   logger
   fi
   cryptsetup --key-file $keyfile luksOpen $dev $cryptname   logger
   [ -b $cryptdev ] && echo $options :$cryptdev   true
  
 case "$volume" in
   # LUKS-encrypted volumes
   lacie-keys)
       autoluks lacie-keys -fstype=ext4,ro,noatime,nodev,noexec,nosuid
       ;;
   lacie-keys-rw)
       autoluks lacie-keys -fstype=ext4
       ;;
   lacie-backups)
       autoluks lacie-backups -fstype=ext4
       ;;
   # Non-encrypted volumes
   cd)
       echo -fstype=iso9660,ro,nodev,nosuid            :/dev/cdrom
       ;;
   floppy)
       echo -fstype=auto,sync,nodev,nosuid             :/dev/fd0
       ;;
 esac

The astute reader will have noticed that the keys volume is mounted either read-only or read-write with the same mechanism. This volume contains SSH keys that I use quite regularly, that filesystem is going to be mounted and unmounted very often. USB keys, and flash memory in general, don't really like repeated writes. Fortunately, the SSH keys are mostly read, and very rarely written to, so I can afford to mount the partition read-only and save the write cycles for when I generate new keys (the mere action of mounting a partition read-write changes it, even if the files in it are never modified). The backups volume is mostly accessed in write mode, and much less frequently anyway, so there's no need for a distinction there. This supposes that the partitions on the USB key appear as /dev/lacie-keys and /dev/lacie-backups. Due to the dynamic naming of devices, you may like to use rules similar to the following in /etc/udev/rules.d/local.

 KERNEL=="sd[a-z]1", ATTRS idProduct =="1027", ATTRS idVendor =="059f", ATTRS manufacturer =="LaCie", ATTRS product =="LaCie iamaKey", SYMLINK+="lacie-keys"
 KERNEL=="sd[a-z]2", ATTRS idProduct =="1027", ATTRS idVendor =="059f", ATTRS manufacturer =="LaCie", ATTRS product =="LaCie iamaKey", SYMLINK+="lacie-backups"

You'll also need to initialize LUKS on the partitions, save the key, make the filesystems, and so on. If you're still with me, you probably know what I'm talking about, then I don't need to explain about that. This setup works for me, but there's no guarantee and so on. Based on a tutorial found on the Debian Administration website (great resource, by the way). Update: I'm told a large part of the script could be replaced with pmount. If you like short scripts, you may want to investigate that too. Update 2: Josselin points out a different solution using Gnome, udisks, gnome-disk-utility and Nautilus. It's worth checking out, for those who like to run all that and click around. I can't test it right now (Gnome/Nautilus currently refuses to mount any USB key, for some presumably transitional reason), but I have no reason to believe it's inherently broken. I just don't like having to do anything when I plug my key or before I unplug it, and I'm not always running a graphical session with a desktop.

For some time, I've had this idea brewing for how the 'net would work without DNS. Initially I was mostly thinking of how to get rid of the dark side of the registrar business, but it appears that maybe it could help with security in general too. It may not be feasible at all, or I may have forgotten extremely crucial points. I'm not a DNS guru, so bear with me and consider this as a thought experiment. This is, however, a proposal for how the Internet could be made truly decentralized, or acentric. Made so, because it currently is very centralized on at least two counts: the DNS is an inherently hierarchical structure of authority; and so is the situation with the SSL certificates used for securing websites. Let's start with DNS, if you will. Basically, the idea is to replace the current Domain Name System, which is centralised in nature (even if subdomains can be delegated) with a fully decentralised system with no central authority. The point of current DNS is to map names to IP addresses in a unique way. It currently works by registering a this name is managed by such-and-such server, ask for details at such-and-such IP address message. Of course, since there's only one namespace (or a limited number thereof, if you consider that TLDs are different namespaces), there needs to be a central authority to resolve conflicts, because some names will be more popular than others. I propose to start with a cryptographic key pair (think GPG key). Anyone can create their key pair; we assume that it's extremely unlikely for two people to generate the same keys, and that it's also very hard for two people to generate keys with the same fingerprint. By (being the only one) knowing the secret part of a key, I am the authority on what can be signed with it. I could for instance sign records that say that www points to this set of IP addresses, and smtp points to that other address. Within my area of authority, which is to say within my domain. Which is identified by the fingerprint of my key pair. What do we have so far? A mechanism that allows in a decentralised way to verify who's authoritative on the F5566852EB92BD779CF137190EA756B5144843F5 domain . In DNS terms, more probably F55...3F5.some-tld. Now let's publish the signed records on a decentralised peer-to-peer network. I'm not up to date with the latest trends in that domain, but I think something like Gnutella or Freenet would qualify, since it doesn't need central hubs but only an initial list of nodes, and it does automatic discovery of new nodes. Projects such as the FreedomBox (name likely to be changed) will probably help with that. Anyway, when looking up www.F55...3F5.some-tld, the resolver gets the signed record (and the public part of the key) from the P2P network, checks the signature, and uses the contents of the record to determine the IP address. No central or delegated registry involved at any point. Lesser risk of a single point of failure, which means better resilience against outages. Also, a better resilience against DNS-based censorship, be it promoted by non-democratic states or by greedy corporations. Of course, F5566852EB92BD779CF137190EA756B5144843F5 isn't a terribly easy name to remember or transmit. But the string of hex could be presented to the user in some other form easier to recognize for people without geek super-powers. My favourite would be the bibi-binary representation, but I'm told that the bubble-babble method works equally well. It would still be unreasonable to expect people to remember such names, and especially to type them into their web browsers or e-mail agents. A possible solution to that would be to use a short form, such as 144843F5 (or its bibi-binary equivalent), complemented by a visual representation of the full fingerprint such as the one SSH does. That pair could be displayed on business cards, email signatures, glossy brochures, advertisements and so on, and allow users to just type the strange word or flash the 2D barcode, check that the funny picture matches (in case the strange word returns several results), and get the website they want. Which they can of course bookmark under a name they can remember, probably even the same name that everyone else uses to bookmark that website. And nothing precludes convenient directories mapping names to full URLs (like search engines currently do) as long as users check that the visual representation of the site they go to matches what they've seen elsewhere. That elsewhere could, of course, take advantage of the natural web of trust that emerges in of GPG-using communities. If I'm directed at a website whose key I don't immediately recognize, then I can see who trusts that key to be legitimate (by checking the signatures), and who trusts them, and so on. Ultimately, the decision is mine to make, and mine only. In any case, once I know what URL I want to visit, I can be sure this DNS replacement will give me the right IP address (or addresses). DNS servers replying with advertisement sites instead of correct ones, and DNS-based filtering, will be things of the past. Of course, having the right IP address only solves the attacks on the DNS part. Eavesdropping can also happen by diverting traffic at the IP level. But we have a trustable way to know information related to a domain, so let's use it: the signed record can also contain the fingerprint of cryptographic keys (SSL or SSH or whatever). So we can be sure we're actually talking to the correct server, rather than to some man-in-the-middle spy. With the same guarantees of correctness as the ones for the DNS, backed by the web of trust and out-of-band validation. Again, we get rid of the hierarchy of certification authorities , some of which have been known for helping organizations get at the (supposedly secret) data people exchange over the net. To summarize: with this scheme, if I know the site I want to access is F55...3F5, then I can be certain that a) I get the right IP address for it and b) I'm actually talking to it. And nobody in-between can intercept the traffic, if the site is properly administrated and its private key is kept secure. And from the site admin's point of view, there's no need to trust DNS registrars and SSL certification authorities not to fool around with the data, or blackmail me into paying increasing fees for keeping my URL working or my SSL key certified. There's still the problem of the allocation for the IP addresses themselves, but two major parts of the centralized and hierarchical net can be gotten rid of. There are proposals attempting to fix part of the problems already, but they're incomplete. DNSSEC only solves the authentication of DNS records, not the centralisation problem. Monkeysphere aims at adding a web of trust to the SSL certificates system, but the CA chain problem seems to persist (although I must admit I'm not up to speed with the actual details). This is of course only a rough draft, full of technical considerations yet certainly not exhaustive. I've thought about it though, and I believe there are no theoretical problems with the implementation of such a scheme (apart for the handling of expiration and revocations, probably). The main obstacle to mass-adoption I can envision is the UI part, and the entrenched habit of just typing a keyword into a search engine and implicitly trusting its results, then blindly trusting the website itself; but even if mass-adoption isn't reached, I can't see anything wrong with giving those who want to be careful a more secure Internet.

A long time ago, when I was first introduced to email, I was using the Mail program from Unix. I quickly converted to Elm, then Mutt, which were both better in terms of interface. Then I found out about Gnus, and I wouldn't dream of letting it go now. However, Gnus has started showing its age several times, and several times have I needed to upgrade the way I was using it: first because I needed to sort and split email, then because I took the sorting out of Gnus and into Procmail for more advanced filtering (including spam filtering), then because I switched to storing the emails on an IMAP server so I could read them remotely from several computers. My setup as of a few days ago was functional, but since I have grown over time to splitting emails into several hundred folders, checking for new messages was becoming more and more boring. So it's time to jump in with all the cool kids and switch to a modern solution: still Gnus of course, but with Dovecot, OfflineIMAP for synchronisation, and let's add email searches into the mix while we're at it. My web searches didn't turn up a simple step-by-step HOWTO, but I assembled bits from different places, and here's my attempt at documenting my new setup. Goals

• Gnus
• Offline operation
• Searching through emails
• and make it fast!

Assumptions

• Email gets delivered (by SMTP or fetchmail or whatever) to a remote server, which can be accessed through IMAP.
• The client is running some sort of standard Unix-like system; Debian GNU/Linux for me, but it should also work with a BSD or Solaris or something else.
• There is enough space on the client to have a copy of the whole email store (or at least of the folders you want to use with this setup Gnus can of course access the others through the slower IMAP server).

Dovecot setup We'll use Dovecot as a local IMAP server. And since we're lazy, we'll access it over a pipe, and dispense with the network part.

• Install Dovecot (aptitude install dovecot-imapd on Debian systems and related).
• Disable its automated running unless you need it for other purposes; on Debian, set ENABLED=0 in /etc/default/dovecot.
• Decide on where you'll store your emails locally; typical place would be $HOME/Maildir.
• That's it!

OfflineIMAP setup OfflineIMAP is basically an optimised two-way synchronisation mechanism between two email repositories . We'll use it in IMAP-to-IMAP mode.

• aptitude install offlineimap, or similar.
• You'll need a ~/.offlineimaprc file, with contents based on the following example:

 [general]
 accounts = MyAccount
 pythonfile = .offlineimap.py
 [Account MyAccount]
 localrepository = LocalIMAP
 remoterepository = RemoteIMAP
 # autorefresh = 5
 # postsynchook = notmuch new
 [Repository LocalIMAP]
 type = IMAP
 preauthtunnel = MAIL=maildir:$HOME/Maildir /usr/lib/dovecot/imap
 holdconnectionopen = yes
 [Repository RemoteIMAP]
 type = IMAP
 remotehost = mail.example.com
 remoteuser = jsmith
 remotepass = swordfish
 ssl = yes
 nametrans = lambda name: re.sub('^INBOX.', '', name)
 # folderfilter = lambda name: name in [ 'INBOX.important', 'INBOX.work' ]
 # folderfilter = lambda name: not (name in [ 'INBOX.spam', 'INBOX.commits' ])
 # holdconnectionopen = yes
 maxconnections = 3
 # foldersort = lld_cmp

• You will of course need to replace the remotehost, remoteuser and remotepass values with your own. If you want to synchronize only a subset of the IMAP folders, uncomment and adapt one of the folderfilter lines; the lambda can be any Python code you like, I guess.
• Run offlineimap. See the emails coming in and being replicated to your local store.
• Run MAIL=maildir:$HOME/Maildir /usr/lib/dovecot/imap. You should get * PREAUTH [CAPABILITY ... STATUS] Logged in as your-login. Type * LIST "" * in there. You should see a list of all the folders.
• The .offlineimaprc mentions a .offlineimap.py, which is where we're going to store some additional code used by OfflineIMAP. Here's a sample:

 # Propagate gnus-expire flag
 from offlineimap import imaputil
 def lld_flagsimap2maildir(flagstring):
   flagmap =  '\\seen': 'S',
              '\\answered': 'R',
              '\\flagged': 'F',
              '\\deleted': 'T',
              '\\draft': 'D',
              'gnus-expire': 'E' 
   retval = []
   imapflaglist = [x.lower() for x in flagstring[1:-1].split()]
   for imapflag in imapflaglist:
       if flagmap.has_key(imapflag):
           retval.append(flagmap[imapflag])
   retval.sort()
   return retval
 def lld_flagsmaildir2imap(list):
   flagmap =  'S': '\\Seen',
              'R': '\\Answered',
              'F': '\\Flagged',
              'T': '\\Deleted',
              'D': '\\Draft',
              'E': 'gnus-expire' 
   retval = []
   for mdflag in list:
       if flagmap.has_key(mdflag):
           retval.append(flagmap[mdflag])
   retval.sort()
   return '(' + ' '.join(retval) + ')'
 imaputil.flagsmaildir2imap = lld_flagsmaildir2imap
 imaputil.flagsimap2maildir = lld_flagsimap2maildir
 # Grab some folders first, and archives later
 high = ['^important$', '^work$']
 low = ['^archives', '^spam$']
 import re
 def lld_cmp(x, y):
   for r in high:
       xm = re.search (r, x)
       ym = re.search (r, y)
       if xm and ym:
           return cmp(x, y)
       elif xm:
           return -1
       elif ym:
           return +1
   for r in low:
       xm = re.search (r, x)
       ym = re.search (r, y)
       if xm and ym:
           return cmp(x, y)
       elif xm:
           return +1
       elif ym:
           return -1
   return cmp(x, y)

The first part of this file adds a new flag that OfflineIMAP will propagate back and forth. By default, only the standard IMAP flags are propagated; we also want to synchronize the gnus-expire flag that Gnus uses to mark expirable articles. It's a hack, but it works for now (maybe someday OfflineIMAP will propagate all the flags it finds?). The second part of that file can be dispensed with (and won't be used unless the foldersort option is uncommented in .offlineimaprc): it's only there to ensure that some important folders are propagated first, and some others go last. I don't know exactly how they are sorted by default, but I'd like the most important ones to come first, so I can start reading them while the archives and the spam are still being fetched. Gnus configuration

• You need to set up a select method for the local IMAP server. The method will be nnimap, the address can be whatever you like (it's just a name anyway), and you only need two options: nnimap-stream, set to shell, and imap-shell-program, which you set to "MAIL=maildir:$HOME/Maildir /usr/lib/dovecot/imap".
• Then browse the server, subscribe to the folders, and so on.
• From time to time, run offlineimap from a terminal, so you get the new messages, delete the old ones, and so on.
• What's that? You hate running stuff by hand? So do I. To run OfflineIMAP in a loop, uncomment the autorefresh option in .offlineimaprc. The value is the duration (in minutes) between two runs.
• Even starting OfflineIMAP is too much work? offlineimap.el to the rescue! Grab it, put it where Emacs will find it, and add the following to your .emacs:

(require 'offlineimap)
(add-hook 'gnus-before-startup-hook 'offlineimap)

Bonus: email searches The simple way:

(require 'nnir)

This goes in your .gnus. Then your group buffer will get a new command. Mark some folders with #, then M-x gnus-group-make-nnir-group (or use the G G shortcut), and type in a set of keywords. This search is performed by the IMAP server (Dovecot), which may or may not be very efficient, especially if you select many folders. Bonus+: email searches, faster The real cool kids use Notmuch nowadays, at least for email indexing and searching. It's fast, it allows complex queries, and it's generally cool. The downside is that it uses up quite some disk space for its indices, in addition to the actual emails. For that reason I'll keep it to my main computer, and I'll stick to nnir on my laptop (which has the same setup apart from that).

• aptitude install notmuch, yada yada.
• notmuch setup, tell it where you stored your Maildir. If you're only going to use Notmuch for searches, I suggest setting tags= to an empty value in .notmuch-config afterwards. We don't want Notmuch to intrude.
• notmuch new will get it indexing the messages you already have.
• Of course, being lazy is being cool, so let's uncomment the postsynchook line from .offlineimaprc.
• Now add (require 'notmuch) to your .gnus. Also (define-key gnus-group-mode-map "GG" 'notmuch-search), for the shortcut.
• But seeing just the message that matches is not enough, sometimes we want the whole thread. Here's a snippet of Lisp for your .gnus, based on Tassilo Horn's configuration. Do a Notmuch search, enter one of the results, type C-c C-c, you'll get transported to the folder where that message was, with the context. Note this requires code from org-mode, so you might need to install that.

(require 'notmuch)
(add-hook 'gnus-group-mode-hook 'lld-notmuch-shortcut)
(require 'org-gnus)
(defun lld-notmuch-shortcut ()
(define-key gnus-group-mode-map "GG" 'notmuch-search)
)
(defun lld-notmuch-file-to-group (file)
"Calculate the Gnus group name from the given file name.
"
(let ((group (file-name-directory (directory-file-name (file-name-directory file)))))
  (setq group (replace-regexp-in-string ".*/Maildir/" "nnimap+local:" group))
  (setq group (replace-regexp-in-string "/$" "" group))
  (if (string-match ":$" group)
      (concat group "INBOX")
    (replace-regexp-in-string ":\\." ":" group))))
(defun lld-notmuch-goto-message-in-gnus ()
"Open a summary buffer containing the current notmuch
article."
(interactive)
(let ((group (lld-notmuch-file-to-group (notmuch-show-get-filename)))
      (message-id (replace-regexp-in-string
                   "^id:" "" (notmuch-show-get-message-id))))
  (if (and group message-id)
      (progn 
  (switch-to-buffer "*Group*")
  (org-gnus-follow-link group message-id))
    (message "Couldn't get relevant infos for switching to Gnus."))))
(define-key notmuch-show-mode-map (kbd "C-c C-c") 'lld-notmuch-goto-message-in-gnus)

Wrap-up This setup can be replicated on several computers, of course. I have it on two, and there's no reason I couldn't have more. The flags do get propagated back and forth, including the Gnus-specific expirable flag. Accessing the local Dovecot is much faster than going through the DSL to the master IMAP server, and I'm pretty convinced that OfflineIMAP and its multi-threading is also faster than Gnus is, even talking to the same remote server. The email searching with Notmuch is a nice bonus, especially since they're fast too (and this despite my 8-year-old computer). There are a few minor glitches. I can live with them, but I should let you know anyway.

• offlineimap.el keeps an OfflineIMAP process running (if using autorefresh), which causes Emacs to complain about when you want to exit.
• OfflineIMAP propagates the new folders from the remote server to the local store, but not the other way round. Folders created from Gnus won't be propagated, so they'll only be visible on one computer.

Apart from that, I'm pretty happy with this new setup. So I hope this documentation will be useful to others, so I can spread the happiness around. Send your thanks to the authors of the software involved (Gnus, Dovecot, OfflineIMAP, offlineimap.el, Procmail, and so on). Let's see how many years I'll keep that system!

Another month, another update, but nothing spectacular to be announced in FusionForge-world. We're still working on finishing the transition to the new configuration system, we're testing the migration to a simpler and more flexible set of Apache configuration files, and work is in progress on the RPM packaging. And so on. Possibly the most newsworthy item is the FusionForge presence at next month's Libre Software Meeting in Bordeaux (the RMLL in French). I'll do a FusionForge, one year and a half later talk summarising the status and progress of FusionForge, and there'll also be a *forge devroom where we'll mingle with people interested in all kinds of software forges. Come see us if you're around!

The usual semi-regular bits of news from the FusionForge project. We continue being quite active, with several hundred commits each month. The momentum doesn't seem to stop even after the 5.0 release last month. A large part of the activity stems from the Coclico project, which has several work-packages related to convergence of code across forges (mostly between FusionForge and Codendi). This convergence comes in three flavours:

• some new features are developed in common for both forges; as an example, the Mailman and ForumML plugins recently committed can now run unmodified in FusionForge and in Codendi (or at least that's the goal);
• some features existing only on the Codendi side are ported across to FusionForge; this includes the Codendi widgets system, which allow drag-and-drop customization of some of the web pages, and the Hudson plugin;
• finally, some of the core code is rewritten so that a common API and abstract data model can be used by higher-level pieces of code; the configuration system has been almost completely converted to a simple API, and the role-based access control system has also been rewritten into a clean model that extends both FusionForge's previous RBAC system and Codendi's, so each forge will provide new features for access control (the immediate gain for FusionForge is the ability to grant different permissions to anonymous visitors and to visitors that are logged in even if not members of the current project).

All in all, a fairly busy period for FusionForge. The current trunk is evolving rather fast, with some long-overdue rewrites being underway. Interesting times.

Fourteen months after the renaming of the Free/Open Source code of GForge 4.x to the new FusionForge name, we're pleased to announce version 5.0. As mentioned in the release notes, this is still an incremental step over version 4.8 rather than a revolution, but the changes are important enough, and numerous enough, that we felt it justified to bump the major version number. Major improvements, beyond a host of bugfixes, include:

• a rewrite of the version control integration (with support for Bazaar, Darcs, Git and Mercurial in addition to the traditional CVS and Subversion);
• a much better integration of Mediawiki (one wiki, with its own set of permissions, per project);
• a cleaner database layer, more robust against SQL injections;
• configurable display for the trackers;
• more powerful tracker engine, with configurable workflows;
• a rework of the default theme, with better accessibility.

FusionForge 5.0 now also includes new plugins that were previously only floating around (or completely private):

• projectlabels gives a simple way of adding bits of HTML onto project description pages, so the forge admin can, for instance, highlight a project of the month ;
• extratabs allows a project to define new tabs in its pages, pointing at external resources;
• globalsearch is a first step in the federation of forges concept, whereby a project search can be conducted on several forges at once;
• contribtracker allows a forge to prominently display major contributors to projects, to give them visibility beyond the simple commit logs.

These plugins, as well as a large part of the improvements in the trackers and the rewritten Mediawiki plugin, are a direct consequence of the upstreaming of work having been done in private instances of forges. We're happy to note that this goal of ours (to merge local patches into the central repository when it makes sense) seems to be working well. For the record, this 5.0 release includes work and plugins that were reintegrated from sources such as Alcatel-Lucent, Adullact and AdaCore. This release is also the first to have had the benefit of automated testing during the whole cycle. Coverage isn't 100 % yet, but the existing unit tests and functional tests help us be confident in the quality of the release. We'll keep adding more tests as time passes, of course. Looking back at the initial goals stated when the project started, we seem to be on the right track:

• stable release pushed out: check (this is the third one, not counting minor releases);
• new plugins merged: check;
• automated testing: check;
• external contributions merged: check;
• explicit governance model and release process: sort-of (there's still a cabal, but it's partially documented).

We still need to work on the database schema and the cross-distro part, as well as cross-forge interoperability. The good news is that work is happening on these fronts already. And with almost 2500 commits, we truly seem to have accomplished at least one of the (implicit) goals: to bring development back to a healthy state. And we're far from being out of ideas for the future, so there's a lot of good stuff still cooking!

Here's another quick update on the status of FusionForge. We released version 4.8.3. Nothing earth-shattering, but a collection of bugfixes that had accumulated on the 4.8 branch. If you're running a patched version, you might want to merge. We also published the second release candidate for 5.0. It's not final yet (there have been a few commits on that branch since then), but we're running out of known bugs. We're currently down to zero open bugs targeting 5.0, so the actual release is probably going to happen in a matter of days. 5.0rc2 is currently available in Debian experimental for those who want to test it, and the final 5.0 will be uploaded to unstable, and hopefully migrate to Squeeze in due time. Stay tuned

This is getting old news, and others have blogged about them before I did, but here's my summary of the recent activity in and around FusionForge. The early February meeting was a success, and gathered about twenty people on the first day and a dozen or so on the second day (not planned initially). My impression is that there was a healthy mix of FusionForge hackers, FusionForge users, and people from other forge communities (Codendi, NovaForge, and even one representative from nFORGE, from South Korea). I'm not going to repeat all that was said then, especially since the proceedings are online. Beyond the technical points, I'll just advertise PlanetForge again, since everyone present agreed we had lots to share and that this site would be a good and relatively neutral place. If you're into forges, I recommend joining us in that community. On the purely FusionForge front, news are good too. Most of the major pieces we want to see in the next release (which is probably going to be called 5.0) are in place. The last blocker we had was the merge of the rework of the default theme for better accessibility and easier maintenance and customisability (most of the theming now happens in CSS). This merge has been completed this week, and although there are still a few rough edges, it's mostly done. We'll try to fix most of these rough edges soonish, then start a stabilisation branch towards 5.0, so more experimental work can start again on trunk. For the impatient and the curious, there's a list of new features on the fusionforge.org homepage, and the site is now running code from trunk. Of course, we're eager to get testers for that, which is why I prepared snapshot packages. They are currently stuck in NEW on their way to the official Debian experimental repository due to the renaming of the source package and the introduction of plenty of new binary packages, but they can already be obtained from my unofficial repository at people.debian.org. The packages are built for Debian unstable, but they seem to run just fine on Lenny if you grab mediawiki from backports.org (only required for the Mediawiki plugin, of course), and libnusoap-php and php-htmlpurifier from Debian testing (they don't drag any extra dependencies). I'll end this note by reminding people of the announcement I did three months ago: as of this week, Debian Etch is no longer officially supported security-wise, and so neither is GForge 4.5. As far as I know, I was the last person doing that, and my incentives have gone away on the day Etch ceased to be supported, since it was also the day the Adullact forge finally migrated from Etch with GForge 4.5 to Lenny with FusionForge 4.8. If you're still using 4.5, well I think you should be aware of that. That more or less wraps it up for now. The next announcement is likely to be about a release candidate

Just an update about sgeps, because it seems to have made a small stir (which is more than I expected).

• Yes, I know about emacs foo.gpg. Admittedly I found out while I was developing sgeps, but I kept on my track anyway. The real reason was that I was having fun, but I could also mention that sgeps doesn't store data unencrypted on disk, not even temporarily. I'm not sure about vim foo.gpg. (Update: Joey Hess telle me it does the right thing.) Anyway, I don't want to fire up an editor (or switch to an already opened one) just to get a password.
• I had also found out about pwman too. My script started as pwman.pl but it was renamed later. I like the simplicity of sgeps better, especially the lack of any UI besides the CLI, but pwman is probably good in its own way, but it doesn't fit my usage pattern.
• It seems the Gentoo people don't share my qualms about making very small packages, and apparently sgeps is now packaged for Gentoo Linux.
• Mehdi Dogguy already contributed suggestions (including better error handling) and even a patch implementing sgeps --delete. Thanks to him!
• An anonymous commenter suggests that sgeps should be able to store notes as well as passwords. That wasn't a requirement I initially had, but I won't reject the patch if it comes He also argues that it should be able to push the password into the X11 paste buffer. Again, why not, if it doesn't break anything.

I've been accumulating passwords recently. More than I could remember all in one go. I even got worried that I'd locked myself out of one of my own servers recently. So I decided to play it safe and store the passwords somewhere. However, plain text files, even on an encrypted disk, aren't the most secure plan, so I tried to go shopping for a tool that would store passwords in encrypted files and wouldn't be too inconvenient to use. I found a few (pwsafe, keysafe, keepassx, yapet and so on), but they all seem to be either graphical or using their own encryption scheme and (presumably) storage format. Being rather nervous about long-term data accessibility, I thus decided to roll my own script, that would be as simple as possible while doing just the required amount of work. I call the result sgeps, for simple GnuPG-encrypted password store . Note the initial s: I didn't invent any wheel.

• Data model: a list of key/value pairs (each being a string);
• Storage: serialisation using Perl's built-in Storable module;
• Encryption: the serialised data is GnuPG-encrypted;
• Hopefully secure: no password stored in plaintext files at any time.

The code comments should give an idea of the capabilities of sgeps:

  # Usage: sgeps --create                     to create the store
  #        sgeps --add <key>                  to add a key/value to the store
  #        sgeps --list                       to list existing keys
  #        sgeps --add --overwrite <key>      to replace a key/value

I trust both GnuPG and Perl to stay around for quite some time, so hopefully I can forget even the passwords I use very rarely and still be able to recover them later. Even in the event of a hard drive dying, since the encrypted store can now be backed up and burnt on DVDs. I just need to be careful about my GnuPG key. Interested people can grab sgeps from its Bazaar branch with bzr branch http://bzr.debian.org/users/lolando/sgeps/trunk/ or browse it on the web interface. I don't plan to make a Debian package for a hundred lines of Perl code, but if anyone is interested, feel free to include it in an existing package (moreutils maybe?).

News is slow this month on the FusionForge development front. We're all busy gathering all the things that we want to go into the next release, but there's no big news from the code. However, there is something of interest. You may have heard about the Coclico project, which is an initiative aiming at collaboration and convergence between several forge engines, most notably FusionForge, Codendi and Novaforge. That project was started last October, and it holds regular meetings with its members. The next meeting is scheduled for the 2nd of February in Paris, and we thought we could host an open meeting on the 3rd for non-Coclico members, a bit like the forge meeting we had last year (which is when FusionForge was officially born), but with an emphasis on what Coclico did so far. Since most of the FusionForge hackers are in Western Europe, and several are in Paris (especially if we add those who go to Paris for the Coclico meeting), we thought it would also be a good opportunity to gather for a technical and social meeting. It seems the Coclico open session didn't generate much interest this time (at least, it hasn't so far), so I proposed to hijack the room for this FusionForge meeting, and I didn't hear any objections. I have several themes I'd like to discuss with people, and possibly start implementing during that day:

• database maintenance and schema: unification of the upgrade scripts (including for plugins), cleanup of obsolete stuff, addition of missing constraints, and so on;
• configuration system: my initial prototype didn't raise many objections (at least in its scope), now what to do with the next steps?
• packaging and installation system: what needs to be done to keep the three ways of installation (manual, *.deb, *.rpm) in sync with as little work as possible?
• permissions system: clarification of what happens currently, ideas for evolution;
• plugins and interaction with external software: do we lack stuff that would make this easier?
• roadmap, long-term plans, this sort of things;
• other things that users may want to discuss with hackers?
• possibly drink a beer or two;

and so on. These are in no way specific to FusionForge, and in fact I think it would be great if hackers/users of other forges were present, because we could benefit a great deal from their experience and plans. But if we find ourselves amongst FF people only, I think these would be good to discuss, possibly write some code for, and go home with a clearer picture of where our efforts should focus in the near future. I'd therefore like to invite interested people to mark the 3rd of February on their agendas. The meeting will take place in Issy-les-Moulineaux (near Paris, within reach of the tube). If you're interested, please get in touch with us (#FusionForge on the FreeNode IRC network, or the fusionforge-general mailing-list), so we can have a rough estimate of how many people to expect. The meeting room is provided by France T l com, and they're probably going to need numbers if not names. Further details will be announced when known.

I normally don't relay security announces for GForge or FusionForge on this blog, but I will make an exception this time: Alain Peyrat found several places in the code with insufficient input sanitizing, which can cause cross-site scripting vulnerabilities (CVE-2009-3303). It's been fixed in the 4.7 and 4.8 branches as well as the trunk of FusionForge (and in Debian Sid and Squeeze), and updated Debian packages for GForge 4.5 and 4.7rc2 have been released for users of the Etch and Lenny distributions. The reason I make an exception for announcing this here is to remind people that I appear to be the only one maintaining code for GForge 4.5. I do that for two reasons: first, because I'm the maintainer of the package in Debian, and Debian Etch has GForge 4.5, and Etch is supported for security fixes; second, because I also admin/maintain an instance for a client of mine, so I need to backport the fixes anyway, and making them public is no bother. Both of these reasons are going to vanish sometime in the not too distant future: security support for Etch will end in February, 2010, and I hope to have migrated my client's forge to FusionForge 4.8 by then too. A direct consequence is that I will probably stop maintenance for GForge 4.5 in the coming months (at least I'll stop doing it in my free time). So if you're still using GForge 4.5, you should really consider upgrading to something supported, either GForge AS (free download from the GForge Group) or FusionForge (free as in Free Software). Both have an upgrade path. Obviously I think FusionForge is a better choice, but my position is probably biased.

This month hasn't seen many big changes happen in FusionForge. Notable improvements include an initial search engine for Word files, fixes to the automated builds and tests, and lots of bugfixes. The biggest news is probably the start of the Coclico project, an initiative bringing together developers and users of several existing forges in order to reduce the gap (and ideally unify the codebase across the forks) and work together in some fields where cooperation is important. Subjects include a generalisation of the current identity/permission/authentication models and systems, data exchange and migration, interoperability, integration of agile development methods inside the forge, and better integration with the desktop applications such as IDEs. The participants include NovaForge, Codendi, and of course FusionForge. The project only officially started early this month, but we hope to be able to demonstrate results soon. Business as usual apart from that.

Here's another round of the semi-regular bulletin about FusionForge. First item: FusionForge 4.8.1 was released this week. It's not exactly an important update, but the 4.8 branch had been accumulating fixes over time and we felt that it would be good to push these fixes out. If you don't encounter particular problems, there's probably no need to upgrade in a hurry. A follow-up for the rewrite of the SCM subsystem: I now consider the Bazaar and Git plugins complete. The missing part, in both cases, was a proper integration of a repository browser and the collection of commit statistics; since one of my clients wants to use Bazaar and another one wants Git, both features have been completed recently. The code still lives on a branch based off 4.8 (for people who need a 4.8-based instance), but it's also been pushed into trunk so the next release will have it natively. Another branch I've been working on (for clients) was about making the Mediawiki plugin able to handle one wiki per project rather than one shared wiki. This is now possible with yet another 4.8-based branch, where the wiki creation is completely automated. A nice feature is that the FusionForge identification is used as a basis for Mediawiki, with different groups on the wiki depending on project membership and role in the forge. That allows specifying wiki permissions in a simple way, for instance to say that only project members can create new pages, authenticated users can only edit existing pages, and non-authenticated users are read-only. This code will be pushed to trunk in the coming weeks. Thanks to Alain Peyrat, we now have a buildbot running Hudson for unit tests and a few other things. The coverage isn't complete yet, but we hope to increase it as time passes. It's already proven useful, by ensuring at least correctness of PHP syntax, encoding and line-endings. I think that's about it for this time. Business as usual.

Hot from the oven: FusionForge 4.7 was just released. The release notes follow. If you have local enhancements based on a previous version of GForge, now is the time to port them to the 4.7 codebase and submit them, so they can be merged in time for the next version! Release notes for FusionForge 4.7 This is the first public release of FusionForge. FusionForge is based on GForge, and started as an identical copy, with only a name change to avoid confusion with the proprietary versions of GForge (known as GForge Advanced Server or GForge AS). As such, it benefits from mature code and known-good infrastructure, and builds on it for the future. This 4.7 release is focused on bringing the recent evolutions out to the community in an official stable release. This should provide a solid base as a starting point for community-based development, making it easier for enhancements to be maintained. The FusionForge name was chosen to reflect this: this is a community effort, and we hope to hear about your improvements. Contributing these improvements would make their future long-term maintenance easier for everyone. Major changes since previous versions (of GForge) include:

• Support for PHP5.
• Support for PostgreSQL 8.x.
• Translations are now managed by gettext.
• Support for several configurations running on the same code.
• Improved security, no need for PHP register_globals.
• Available as full install CD.
• New wiki plugins (using MediaWiki or phpWiki).
• New online_help plugin.
• New phpwebcalendar plugin.
• New project hierarchy plugin.

Things to keep in mind when installing:

• FusionForge is based on GForge, and the renaming is quite recent. So the code still contains lots of references to GForge. This will be fixed as time passes.
• Full text search using the primitives provided by PostgreSQL 8.3 isn't quite complete yet.
• Not all plugins are packaged for all distributions yet.

Things to keep in mind when upgrading:

• Since internationalisation was changed from a hand-made system to standard gettext, locally customised translations will no longer override standard ones. This will be addressed in a future release.

For more up-to-date information, please visit http://fusionforge.org/ or http://fusionforge.fusionforge.org/ -- you can even join us on IRC from there! -- The FusionForge development team

It's been too long since the last blog post about FusionForge, but we haven't sat on our hands and lots of things happened. This is a brief summary:

• About 900 commits in total over the trunk and the two currently active branches.
• Three minor bugfix versions were released, and the currently stable version is 4.7.3.
• A branch has been started to stabilise the code in prevision for the upcoming 4.8 release. Two release candidates have been published already, the final version should happen in the coming weeks.
• Feature-wise, 4.8 will add a tag cloud for project classification, better reporting for downloads, and a new version of the phpwiki plugin. It will also include the beginnings of an automated test suite.
• 4.7.3 is in Debian unstable, 4.7.2 in testing, 4.8rc1 is in experimental. Don't be fooled by the packages still being called gforge, it's just that they haven't been renamed to fusionforge yet.

Not bad for a start, eh? But we've also kept true to our promise and started merging lots of things from various local branches and patches. Among what's been committed to trunk so far are the following features, coming from various existing or upcoming instances of GForge/FusionForge.

• A script for migrating data from an existing GForge AS instance to FusionForge. It's not a turn-key solution, and it requires some manual intervention for working, but it does most of the job.
• New extratabs plugin, allowing project administrators to define new tabs for their projects, pointing at arbitrary URLs.
• New projectlabels plugin, allowing site administrators to define labels as chunks of arbitrary HTML and stick that HTML onto some projects (useful for "project of the month" stuff).
• Refreshed globalsearch plugin, allowing cross-forge project searches.
• Command-line data injectors to populate a FusionForge database with data coming from another system.
• A more consistent automated self-test suite using Selenium.

Other great things are afoot, and they'll be described here in due time.

Quick heads-up about FusionForge. The main news of course is that 4.8 has been released upstream (and uploaded to Debian experimental). We'll keep fixing major bugs on that branch of course, but our focus is now on trunk. We're finding it tedious to deal with legacy code, so one of the goals we have now is to clean up the codebase to bring it more in line with good practice. That's going to take some time, though, because there's lots of code. Some of that code, however, seems unused (it's been broken for some time without anyone complaining), so it's likely that we'll deprecate and/or remove bits of code unless someone steps forward to maintain it (or at least bring it into shape). In particular, we're looking at the MySQL support (which hasn't been maintained for years) and some of the old visual themes which are going to require some work to keep working with some changes we're planning in the way the pages are displayed. This should make maintenance easier for the implementation or integration of new features down the line. Which will be the subject of a future post, when a currently undercover French Forge Cabal actually starts producing concrete results. Watch this space.

I have a problem with Debconf, but it's far from specific to Debconf. If anything, it's specific to me. I have a short term memory for people, and I tend to forget faces, names and nicknames. And their mappings. Which means that people I haven't seen in a few years tend to get blank looks, puzzled frowns and/or awkward greetings from me. Sometimes I know the face but it takes me a while to put a name on it, sometimes a nickname pops up, sometimes just a feeling that I've met a person for such-and-such occasion in such-and-such place. Sometimes I get it all rushing back at me after a few minutes. This has happened a few times already at this Debconf, and is likely to happen again, except for people who'd be hard to forget (colourful shirts, memorable hairstyles and exuberant personalities tend to stick in my memory). This is bothering me as much as you, and I would like to apologise to all people I'm likely to offend. Sorry, all of you. Whoever you are.

Welcome to this month's FusionForge news batch. I did a presentation of FusionForge at the Libre Software Meeting (Rencontres Mondiales du Logiciel Libre, in French) earlier this month, to explain where we come from and where we hope to go. Many people attended despite the talk being early on the morning following the formal dinner, and the questions showed interest, which is encouraging for the project as a whole. I don't think the talk has been recorded, but the summary and slides are available on the RMLL website. The big news, though, is that I'm currently at the Debian Conference, Debconf, and that I also attended Debcamp before that. Debcamp is a very productive get-together of developers from all across Debian, and I took the opportunity to get help from them. I spent the first few days refactoring some of the code that was duplicated between the CVS and Subversion plugins, and the result is that version control plugins are now much easier to implement. Case in point: I managed to get the attention of a few users of other tools, and since they only had to implement small specific parts, we now have almost complete plugins for Bazaar, Darcs and Git, and Mercurial will probably follow. CPOLD was done too, but mostly as a proof of concept. If you're around, come and see me, we'll finish the support for your favourite tool together. Or even start it (I haven't started on Arch and Monotone for lack of perceived interest, but I'm quite open to these tools too). In both cases, I promise it won't take long. This code currently only lives on a temporary branch based off FusionForge 4.8, but I'll port that to trunk and commit it in the coming weeks.

Executive summary To avoid confusion with the proprietary versions of GForge (known as GForge Advanced Server, GForge Express Edition and GForge Community Edition), the free/libre/opensource codebase will from now on be separately maintained under the name FusionForge by the main developers of the free GForge 4.x codebase. Since this is mostly a renaming, the migration path for existing users will be smooth. Longer version, with details After the initial forking from the Sourceforge codebase, the development of GForge has long been hosted, and many enhancements directly developed, by the GForge Group (GForge, LLC), with regular contributions from outsiders. The results of these evolutions were public and free, subject to the GNU GPL. In parallel, the GForge Group wrote a proprietary re-implementation of GForge, which it sold under the name "GForge Advanced Server", or "GForge AS" for short. This re-implementation added some features for "the enterprise", but was not contributed wholesale to the GForge codebase under a free license. Although some of the features were contributed to the public, the GForge Group concentrated its efforts on its (proprietary software) business model, with more versions appearing, such as "GForge Express Edition" and more recently "GForge Community Edition". As a result, it became increasingly harder for the public to know which version was which without doing extensive research (indeed, some users mistakenly installed one version instead of the other). A consequence was that the free software codebase suffered from a loss in visibility, which lowered its momentum to the point that there haven't been any moderately important releases since the (currently stable) 4.5.x series was announced in late 2005. So, in order to clarify things, avoid further confusion, and regain some of the lost momentum, it was decided by a group of leading contributors that the free software version of the GForge codebase would from now on be developed under the FusionForge name, and its development would be hosted on FusionForge.org. So is this a fork? Well, we don't know yet. It could arguably be called one, since we're taking the code and running away with it under a new name. However, we believe it's not a fork unless both roads continue their own way (more of a oddly-shaped bend). What happens to the GForge codebase developed by the GForge Group at gforge.org remains to be seen, although for the sake of our users we will backport security fixes to the gforge.org Subversion repository (at least for the 4.5.x series and the unreleased 4.6 and 4.7 pre-series) for some time. The bulk of the development will move on to FusionForge and the repositories at FusionForge.org, though, and users are encouraged to migrate at their own pace. Since we're basically continuing the evolution rather than starting from scratch, the migration path should be rather smooth. So why the FusionForge name? Because there were actually lots of locally-patched versions of GForge (and Sourceforge), and we felt it was a waste of resources that should be fixed. It seems many people and organisations took these codebases at some point in time and evolved them for their own needs. Sadly, many of the changes were not contributed back or even published, so lots of efforts were duplicated. Fortunately, many of the people managing these locally-patched forges are now realising that "out-of-tree" patches and features require quite some manpower to maintain. Some formal inter-project discussion is already taking place, and we hope to achieve actual merging of most of the interesting features that have been developed here and there into a common base that can be reused locally with minimal changes. We'd like to "un-fork" as much as possible. We also expect that, by using standard components and tools, we'll facilitate the work of potential contributors, thereby reducing the risk of a new era of fragmentation. And who are we anyway? We're Christian Bayle, Roland Mas and Alain Peyrat, long-time contributors to GForge and responsible for over 95% of the commits over the past two years, as well as a few relative newcomers. Christian and Roland have been maintaining the Debian packaging since the "Debian-SF" era, and Alain has been focusing on code quality. The three of us have, for various reasons, a vested interest in maintaining a lively codebase in a healthy ecosystem. What are our plans? Our short-term goals, as currently planned, include:

• pushing a stable FusionForge release out of the door;
• cleaning up and auditing the database schema to ensure consistency and increased performance;
• merging in new plugins, in particular for new version control systems;
• continuing streamlining the installation process to make it more portable across distributions.

Longer term goals are less well defined, but we're thinking about the following:

• increasing code quality by the use of modern automated tools;
• encouraging a lively stream of external contributions, to reduce the gap between the official version and the locally-patched ones;
• defining an explicit governance model and release process for the FusionForge project;
• as a consequence, a more frequent and predictable release schedule;
• increasing data portability and interoperability with other forges, to reduce lock-in for users and projects.

Some of these items should be facilitated by our switch to a distributed version control system and a new coordinated workflow. Also, the Debian i18n team has been kind enough to offer to host our translation effort on their Pootle server, which means translators will have a much easier time doing their job. We hope to hear from users and contributors alike in the near future. For more information, we can be reached via our fusionforge-general mailing-list (see our lists), which is also suitable for general discussions. We can also be found on IRC (#fusionforge on the Freenode network).